Inductive Learning with BCT

BCT (Binary Classification Tree) is a system that learns from examples and represents learned concepts as a binary polythetic decision tree. Polythetic trees differ from monothetic decision trees in that a logical combination of multiple (versus a single) attribute values may label each tree arc. Statistical evaluations are used to recursively partition the concept space in two and expand the tree. As with standard decision trees, leaves denote classifications. Classes are predicted for unseen instances by traversing appropriate branches in the tree to the leaves. Empirical results demonstrated that BCT is generally more accurate or comparable to two earlier systems. The workshop version of this paper is in Proceedings of the Sixth International Workshop on Machine Learning (pp. 104-108), June 30 July 2, 1989, Ithaca, NY: Morgan Kaufmann. INTRODUCTION BCT (Binary Classification Tree) is a system that learns from examples. Given a set of preclassified instances, it searches for logical descriptions that accurately represent these classes. BCT is an attempt to unify valuable features exhibited by similar systems. As in AQ (Michalski, 1973) BCT extensively searches for concept descriptions that are consistent with the training examples. Like ID3 (Quinlan, 1986) the system partitions the concept space recursively to identify the most effective class descriptions. Similar to CN2 (Clark & Niblett, 1989) it uses a polythetic approach (based on multiple attribute values instead of single ones) in evaluating the quality of concepts. Moreover, noise-free examples can rarely be obtained and are not assumed during the inductive process. Most importantly, statistically-sound heuristics are incorporated in BCT to guide the search in an accurate and precise manner. In many ways BCT can be viewed as a combination of ID3 and CN2: the concept space is hierarchically partitioned using statistical measures to guide CN2-like searches at each tree level. In this paper we describe BCT in terms of its knowledge representation, learning operators, and heuristic measures. Empirical comparisons reveal that BCT compares favorably with ID3 and CN2. Finally, related systems and directions for future work are discussed. KNOWLEDGE REPRESENTATION BCT represents concept knowledge as a binary polythetic decision tree. Each tree node is a complex, which is a combination of attribute values, called selectors, in conjunctive normal form (CNF). For example, [(surface = hard) and (color = red or blue)] and [(color = red)] are complexes and (surface = hard), (color = red or blue), and (color = red) are selectors. The two branches coming out from a node represent matching and non-matching of the node’s complex (without loss of generality, assume that the left branch matches the complex and the right one does not). Essentially, a complex divides the concept space into two, branching on the complex. The leaves of a tree are classes present in the preclassified examples. Thus, all the leaves (classes) are described by conjunctions of complexes or their negations on the corresponding traversed path. For example, the tree